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(54) Method and apparatus for processing a bitstream 



(57) Apparatus (100, 700) for processing a com- 
pressed input bitstream (112, 712) representing an 
information signal. The apparatus processes the com- 
pressed input bitstream in a way that changes the infor- 
mation signal represented by the compressed input 
bitstream, and comprises a layered decoder (114, 
71 4A) that decodes the compressed input bitstream to 
provide a decoded bitstream (116, 716A), a decoded 
bitstream processor (118, 71 8 A) that processes the 
decoded bitstream to provide a modified bitstream (122, 
722A), a layered encoder (124, 724) that re-encodes 
the mocfif ied bitstream to provide a compressed output 
bitstream (126. 726). and a selective interconnection 
path (134, 215. 315. 138; 725. 721. 719. 738). The lay- 
ered decoder includes serially-connected decoding 
modules (e.g., 201, 203, 705. 707). The decoded bit- 
stream processor processes the decoded bitstream in a 
way that modifies the information signal. The layered 
encoder includes serially-connected encoding modules 
(e.g., 301 , 303, 705, 707), each of which corresponds to 
one of the decoding modules in the layered decoder. 
The selective interconnection path selectively connects 
the output of one of the decoding modules to the input 
of the decoded bitstream processor and selectively con- 
nects the output of the decoded bitstream processor to 
the input of the one of the encoding modules corre- 
sponding to the one of the decoding modules. The lay- 
ered decoder may include an element that derives 
bitstream parameters from the compressed input bit- 
stream, and the apparatus may additionally comprise a 
bitstream parameter forwarding path that interconnects 
the layered decoder to the layered encoder. At least one 
of the encoding modules may be connected to the 



parameter forwarding path and may perform an encod- 
ing operation using the bitstream parameters derived 
from the compressed input bitstream. 
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Description 

Field of the Invention 

The invention relates to processing bitstreams, and, 
in particular, to processing compressed bitstreams in a 
way that reduces the resources required to perform the 
processing. 

Background of the Invention 

Bitstreams representing information signals are 
often subject to compression to conserve communica- 
tions bandwidth or signal storage requirements. For 
example, the bandwidth or signal storage required to 
transmit or store a video bitstream representing a 
motion picture can be reduced by a factor of the order of 
100 by applying suitable spatial and temporal compres- 
sion. Such spatial and temporal compression schemes 
include the MPEG-1 and MPEG-2 schemes standard- 
ized by the ISO. 

Production, distribution and display of video bit- 
streams representing video signals often involves 
processing the video bitstream in some way. Such 
processing may modify the video signal represented by 
the video bitstream. For example, the processing may 
apply color correction or a different gamma correction, 
or may apply noise reduction to the video signal. Such 
processing may additionally or alternatively be per- 
formed to convert the video signal from one video 
standard to another. In this, the number of picture ele- 
ments (pixels) in each picture may be changed, and the 
picture rate may additionally or alternatively be 
changed. Moreover, when the video bitstream is com- 
pressed, the orocessing may also modify the bitstream 
characteristic of the compressed video bitstream. For 
example, the l impressed video bitstream may be proc- 
essed to change its bit rate, or to restructure it so that ft 
can be decoded using a decoder having a decoder 
buffer of a different size. 

An example of a conventional apparatus 10 for 
processing a compressed input bitstream is shown in 
Figure 1. In this example, the compressed input bit- 
stream is an MPEG-compliant video bitstream. Similar 
arrangements may be used for processing video bit- 
streams compressed according to other standards. In 
the conventional apparatus 10 shown in Figure 1 f the 
MPEG decoder 14 decodes the compressed input bit- 
stream 12 to recover the video bitstream 16. The video 
processor 18 applies the processing defined by the 
processing selection block 20 to the video bitstream 1 6 
to generate the modified video bitstream 22. The modi- 
fied video bitstream is fed to the MPEG encoder 24 that 
generates the compressed output bitstream 26 by reen- 
coding the modified video bitstream to have bitstream 
characteristics, such as bit rate, buffer size, etc., defined 
by the processing selection block 20. The bitstream 
characteristics of the compressed output bitstream may 



be the same as, or different from, those of the com- 
pressed input bitstream 12. If the processing selected 
by the processing selection block only involves chang- 
ing the bitstream characteristics, the video processor 1 8 

5 simply passes the video bitstream 16 to the MPEG 
encoder 24 without modifying it. 

The encoding operation performed by the MPEG 
encoder 24 on the modified video bitstream 22 is con- 
siderably more computationally intensive than the 

10 decoding operation performed by the MPEG decoder 
14 on the compressed input bitstream 12. For example, 
to encode a video bitstream representing an NTSC 
video signal in real time requires special hardware or a 
computer based on a microprocessor of equivalent 

rs computing power to a 686-type microprocessor. Conse- 
quently, the conventional apparatus 10 is complex and 
expensive. 

Of the operations performed by the MPEG encoder 
24, the motion estimation operation used to determine 

20 the motion vector for each block is computationally the 
most intensive. The number of operations required to 
perform motion estimation can be scaled down by 
reducing the scope of the motion search region, or by 
using a fast motion searqh algorithm, but both of these 

25 approaches inevitably impair video quality. To ensure an 
acceptable video quality, the MPEG encoder 24 will 
encode most sequences using relatively large search 
regions with an efficient, but suboptimal, motion estima- 
tion algorithm notwithstanding the large computational 

30 resources that this approach demands. 

What is needed is the ability to perform the above- 
mentioned processing on compressed input bitstreams 
without the need for special hardware, or using a com- 
puter based on a lower-cost microprocessor. What is 

35 also needed is the ability to perform the above-men- 
tioned processing on compressed input bitstreams 
faster than in real time using currently-available hard- 
ware. 

In video production and distribution environments, 

40 the video bitstream may be subject to a number of differ- 
ent processing operations. Multiple processing opera- 
tions are conventionally applied to the video bitstream 
by concatenating a number of processors. Each proces- 
sor operates on the conventional, pixel-level, video bit- 

45 stream generated by the preceding processor. Such an 
arrangement of concatenated processors would replace 
the video processor 1 8 shown in Figure 1 . 

Using a concatenated arrangement of processors 
to perform multiple processing operations on a video 

so bitstream normally requires that the video bitstream 
operated on by each processor be a conventional video 
bitstream. Processors that operate on compressed or 
partially-compressed video bitstreams cannot be con- 
catenated with processors that operate on conventional 

55 video bitstreams or with processors that operate on 
video bitstreams that are differently compressed. More- 
over, processors that are designed to operate on a con- 
ventional video bitstream, i.e., on an uncompressed 
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video bitstream, require that computational resources 
be expended to decode the compressed input bitstream 
fully before processing, and to fully re-encode the mod- 
ified bitstream generated by the processors. Finally, 
processing systems that precede the processors by a 
conventional decoder and follow the processors with a 
conventional encoder require the compressed input bit- 
stream to be compressed according to the compression 
standard of the decoder, and provide the output bit- 
stream compressed according to the compression 
standard of the encoder. 

Accordingly, what is also needed is a video 
processing system that allows processors that operate 
on bitstreams in different states of compression to be 
concatenated, that can process a compressed input bit- 
stream in any state of compression and that can gener- 
ate a compressed output bitstream in a state of 
compression that is greater than, less than, or the same 
as that of the compressed input bitstream. 

Summary of the invention 

The invention provides an apparatus for processing 
a compressed input bitstream representing an informa- 
tion signal such as a video signal. The apparatus proc- 
esses the compressed input bitstream in a way that 
changes the information signal represented by the com- 
pressed input bitstream. The apparatus comprises a 
layered decoder that decodes the compressed input bit- 
stream to provide a decoded bitstream, a decoded bit- 
stream processor that processes the decoded bitstream 
to provide a modified bitstream, a layered encoder that 
re-encodes the modified bitstream to provide a com- 
pressed output bitstream, and a selective interconnec- 
tion path. The layered decoder includes serially- 
connected decoding modules. The decoded bitstream 
processor processes the decoded bitstream in a way 
that modifies the information signal. The layered 
encoder includes serially-connected encoding modules, 
each of which corresponds to one of the decoding mod- 
ules in the layered decoder. The selective interconnec- 
tion path selectively connects the output of one of the 
decoding modules to the input of the decoded bitstream 
processor and selectively connects the output of the 
decoded bitstream processor to the input of the one of 
the encoding modules corresponding to the one of the 
decoding modules. 

The invention also provides an apparatus for modi- 
fying the bitstream characteristics of a compressed 
input bitstream representing an information signal. This 
apparatus modifies the bitstream characteristics of the 
compressed input bitstream without necessarily sub- 
stantially changing the information signal represented 
by the compressed input bitstream. The apparatus com- 
prises a controller that provides an output bitstream def- 
inition defining the bitstream characteristics of a 
compressed output bitstream; a layered decoder that 
includes a deriving element that derives bitstream 



parameters from the compressed input bitstream; a bit- 
stream parameter forwarding path; and a layered 
encoder that generates the compressed output bit- 
stream. The layered decoder decodes the compressed 
5 input bitstream to provide a decoded bitstream. The bit- 
stream parameter forwarding path is connected to 
receive the bitstream parameters from the layered 
decoder. The layered encoder is connected to the bit- 
stream parameter forwarding path and re-uses the bit- 

w stream parameters derived from the compressed input 
bitstream to re-encode the decoded bitstream in 
accordance with the output bitstream definition. 

The invention also provides a method for process- 
ing a compressed input bitstream representing an irrfor- 

75 mation signal. The method processes the compressed 
input bitstream in a way that changes the information 
signal represented by the compressed input bitstream. 
In the method, the compressed input bitstream is par- 
tially decoded to provide a decoded bitstream. The 

20 decoded bitstream is a partially-encoded bitstream rep- 
resenting the information signal. A modified bitstream is 
generated by processing the decoded bitstream in a 
way that modifies the information signal. The modified 
bitstream is a partially-encoded bitstream. Finally, the 

25 modified bitstream is re-encoded to generate a com- 
pressed output bitstream. 

The invention further provides a method for modify- 
ing the bitstream characteristics of a compressed input 
bitstream representing an information signal. This 

30 method modifies the bitstream characteristics without 
necessarily substantially changing the information sig- 
nal represented by the compressed input bitstream. In 
the method, an output bitstream definition is generated. 
The output bitstream definition defines the bitstream 

35 characteristics of a compressed output bitstream. Bit- 
stream parameters are derived from the compressed 
input bitstream. The compressed input bitstream is at 
least partially decoded to provide a decoded bitstream. 
Finally, the decoded bitstream is re-encoded using the 

40 parameters derived from the compressed input bit- 
stream to generate the compressed output bitstream 
having the bitstream characteristics defined by the out- 
put bitstream definition. 

The invention also provides an apparatus for 

45 processing a compressed input bitstream representing 
an information signal. The apparatus processes the 
compressed input bitstream in a way that changes the 
information signal represented by the compressed input 
bitstream. The apparatus comprises a first compression 

so state translation unit a processor and a second com- 
pression state translation unit The first compression 
state translation unit receives the compressed input bit- 
stream as an input bitstream and changes the compres- 
sion state of the compressed input bitstream to 

55 generate an output bitstream. The processor receives 
the output bitstream of the first compression state trans- 
lation unit, and processes the output bitstream to gener- 
ate a modified bitstream. The processor processes the 
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output bitstream in a way that modifies the information 
signal. The second compression state translation unit 
receives the modified bitstream as an input bitstream 
and changes the compression state of the modified bit- 
stream to generate an output bitstream as the com- s 
pressed output bitstream. 

The first and second compression state translation 
units each include layered encoding/decoding modules, 
an input interface module and an output interface mod- 
ule. The input interface module selectively connects the 10 
input bitstream to one of the encoding/decoding mod- 
ules. The one of the encoding/decoding modules to 
which the input bitstream is connected depends on the 
compression state of the input bitstream. The output 
interface module selectively receives a bitstream from is 
the output of one of the encoding/decoding modules as 
the output bitstream. The one of the encoding/decoding 
modules from which the bitstream is received depends 
on the compression state of the output bitstream. The 
compression state of the output bitstream may be differ- 20 
ent from that of the input bitstream. 

Finally, the invention provides a method for 
processing a compressed input beam representing an 
information signal. In the method, layered encod- 
ing/decoding modules are provided. A f irst set of encod- 25 
ing/decoding modules and a second set of 
encoding/decoding modules are selected from the lay- 
ered encoding modules. The compression state of the 
compressed input bitstream is changed to provide a first 
output bitstream using the first set of encoding/decoding 30 
modules. The f irst output bitstream is processed to gen- 
erate a modified bitstream. The first output bitstream is 
processed by in a way that modifies the information sig- 
nal but generates the modified bitstream in the same 
compression state as the first output bitstream. Finally, 35 
the compression state of the modified bitstream is 
changed using the second set of encoding/decoding 
modules to generate a second output bitstream. 

Brief Description of the Drawings *o 



in which quantizing is performed in the layered 
encoder by re-using quantizing parameters derived 
from the compressed input bitstream. 
Figure 7A is a block diagram of a second embodi- 
ment of an apparatus according to the invention for 
processing a compressed input bitstream. 
Figure 7B is a block diagram showing details of the 
compression state translation units of the second 
embodiment of the apparatus according to the 
invention. 

Detailed Description of the Invention 

In the following description, reference will be made 
to the compression state of an information signal. Bit- 
streams compressed according to MPEG and similar 
compression schemes are the result of a prescribed 
number of layers of compression applied to the original 
information signal. Compressed bitstreams can also be 
generated by applying fewer than the prescribed 
number of layers of compression to the information sig- 
nal. Accordingly, a compressed bitstream representing 
an information signal can be regarded as having a com- 
pression state that depends on the number of layers of 
compression that have been applied to the information 
signal. The compression state of a compressed bit- 
stream can be increased by applying additional layers of 
encoding operations to the compressed bitstream, and 
can be decreased by applying layers of decoding oper- 
ations to the compressed bitstream. 

A first embodiment 100 of a compressed bitstream 
processing apparatus according to the invention is 
shown in Figure 2. The example shown is adapted for 
processing an MPEG-compfiant video bitstream, but 
can easily be adapted for processing other types of 
compressed bitstreams representing information sig- 
nals. The compressed bitstream processing apparatus 
100 is composed of four main units, the layered decoder 
1 14, the decoded bitstream processor 1 18. the layered 
encoder 124 and the bitstream parameter forwarding 
path 128 that interconnects the layered decoder and the 
layered encoder. In addition, if the compressed bit- 
stream processing apparatus is designed to allow a 
user to select or def ine the processing operation per- 
formed, the apparatus additionally includes the 
processing selection module 120 that receives instruc- 
tions from the user and controls the processing per- 
formed by the layered decoder, the processor and the 
layered encoder. Otherwise, the processing selection 
module controls the processing performed by the lay- 
ered decoder, the bitstream processor and the layered, 
encoder so that the compressed bitstream processing 
apparatus performs a predetermined processing opera- 
tion on the compressed input bitstream. 

The layered decoder 1 1 4 decodes the compressed 
input bitstream 112 to generate the decoded bitstream 
116 and passes the decoded bitstream to the decoded 
bitstream processor 118. The layered decoder gener- 



Figure 1 is a block diagram of a conventional appa- 
ratus for processing a compressed input bitstream. 
Figure 2 is a block diagram of a first embodiment of 
an apparatus according to the invention for 45 
processing a compressed input bitstream. 
Figure 3 is a block diagram shewing part of the first 
embodiment of the apparatus according to the 
invention in more detail. 

Figure 4 is a block diagram that illustrates how bit- so 
stream parameters derived from the compressed 
input bitstream are re-used in the layered encoder 
in the apparatus according to the invention. 
Figure 5 is a fbw chart that illustrates a first way in 
which quantizing is performed in the layered ss 
encoder by re-using quantizing parameters derived 
from the compressed input bitstream. 
Figure 6 is a flow chart that illustrates a second way 
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ates the decoded bitstream in the compression state 
required by the decoded bitstream processor 118. For 
example, when the decoded bitstream processor is a 
conventional bitstream processor that operates on a 
conventional video bitstream, the decoded bitstream is 5 
a conventional video bitstream. Alternatively, the 
decoded bitstream processor may operate on a par- 
tially-compressed video bitstream, such as a bitstream 
composed primarily of DCT coefficients. In this case, 
the decoded bitstream is composed primarily of DCT 10 
coefficients. 

The layered decoder 114 additionally derives from 
the compressed input bitstream various parameters that 
will be called bitstream parameters. The bitstream 
parameters are parameters that are extracted from the 15 
compressed input bitstream 112, or that are derived by 
the layered decoder in the course of decoding the com- 
pressed input bitstream. In the following description, the 
term derived will be understood to encompass either of 
these possibilities. The bitstream parameters collec- 20 
tively define each picture of the video bitstream repre- 
sented by the compressed input bitstream, and include 
coded, quantized DCT coefficients, motion vectors, 
quantizing tables, macroblock-type information, and 
header information. The header information includes 25 
statistical information such as the number of bits per 
block and the number of bits per picture. The layered 
decoder forwards the bitstream parameters to the lay- 
ered encoder 124 through the bitstream parameter for- 
warding path 128. 30 

The decoded bitstream processor 118 processes 
the decoded bitstream 1 16 it receives from the layered 
decoder 114, and passes the resulting modified bit- 
stream 122 to the layered encoder 124. The layered 
encoder is also connected to the bitstream parameter 35 
forwarding path 128 from which it receives the bitstream 
parameters derived from the compressed input bit- 
stream 112 by the layered decoder. The layered 
encoder generates the compressed output bitstream 
126 by wholly or partially re-encoding the modified bit- 40 
stream. In this embodiment, the extent of the re-encod- 
ing performed by the layered encoder is the same as the 
extent of the decoding performed by the layered 
decoder. 

The layered decoder 114 operates only to decode 45 
the compressed input bitstream 112 to the state of com- 
pression required by the decoded bitstream processor 
118. This state of compression is less than that of the 
compressed input bitstream 112, but may be greater 
than that of a conventional video bitstream. In this so 
embodiment, the modified bitstream 122 is in the same 
state of compression as the decoded bitstream 116. 
The layered encoder 124 re-encodes the modified bit- 
stream re-using at least some of the bitstream parame- 
ters derived from the compressed input bitstream 112 ss 
by the layered decoder 114. Re-using the bitstream 
parameters saves the layered encoder from expending 
computational resources to calculate new parameters 
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equivalent to the bitstream parameters that are re-used. 
Consequently, the compressed bitstream processing 
apparatus 100 according to the invention can perform 
the desired processing of the compressed input bit- 
stream using a smaller scale of hardware or a less pow- 
erful computer or digital signal processor than a 
conventional compressed bitstream processing appara- 
tus. Alternatively, compared with a conventional appara- 
tus, the compressed bitstream processing apparatus 
according to the invention can perform the desired 
processing at a faster rate, e.g., faster than real-time, 
using given hardware or a given computer. 

The compressed bitstream processing apparatus 
100 will now be described in more detail. In particular, 
the layered decoder 114 and the layered encoder 124 
will be descrtoed in detail with reference to Figures 2 
and 3. 

The layered decoder 1 14 will be described first. The 
layered decoder receives the compressed input bit- 
stream 112, partially or fully decodes the compressed 
input bitstream, and supplies the resulting decoded bit- 
stream 116 to the decoded bitstream processor 118. 
The layered decoder also derives bitstream parameters 
from the compressed input bitstream and supplies the 
bitstream parameters to the layered encoder 1 24 via the 
bitstream parameter forwarding path 128. In the exam- 
ple shown, the compressed input bitstream and the 
compressed output bitstream 1 26 are both MPEG-com- 
pliant video bitstreams. 

Like a conventional MPEG decoder, the layered 
decoder 114 includes a number of serially-connected 
decoding modules that operate together to decode the 
compressed input bitstream 112. Unlike a conventional 
MPEG decoder, the layered decoder can stop decoding 
the compressed input bitstream at the completion of 
decoding by any one of the decoding modules so that 
the compressed input bitstream is decoded only to the 
compression state required by the decoded bitstream 
processor 118. The decoding module at which the lay- 
ered decoder stops decoding depends on decoding 
control data supplied by the processing selection mod- 
ule 120 via the control path 132. When the decoding 
control data set the layered decoder so that all the 
decoding modules operate to decode the compressed 
input bitstream, the layered decoder decodes the com- 
pressed input bitstream to generate the decoded bit- 
stream 116 as a conventional video bitstream. The 
pictures of a conventional video bitstream are in presen- 
tation order and the bitstream includes a pixel value for 
each pixel of each picture. When the decoding control 
data set the layered decoder so that fewer than air the 
decoding modules are selected to decode the com- 
pressed input bitstream. the compressed input bit- 
stream is only partially decoded. In this case, the 
decoded bitstream 1 16 is not a conventional video bit- 
stream, but is partially compressed. For example, the 
decoded bitstream may comprise a set of DCT values 
for each block of each picture. 
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Since the layered decoder 1 14 can stop decoding 
the compressed input bitstream at the completion of 
decoding by any one of the decoding modules, the out- 
put of each decoding module is not only connected to 
the input of the following decoding module, but is also 5 
connected to the output bus 134 that feeds the output 
selector 215. The output selector will be described in 
more detail below. 

The decoding modules that constitute the layered 
decoder 1 14 are based on corresponding modules of a to 
conventional MPEG decoder. It is assumed that the per- 
son of ordinary skill in this art is familiar with the struc- 
ture and operation of such conventional modules, so the 
decoding modules will only be described briefly. The 
decoding modules include the decoder buffer 201 . the is 
demultiplexer 203, the inverse variable-length coder 
205. the inverse quantizer 207, the inverse discrete 
cosine transform module 209, the motion compensator 
21 1 and the spatial bitstream output module 213. 

The compressed input bitstream, a constant bit-rate 20 
video stream in this example, is temporarily stored in 
the decoder buffer 201. The compressed input bit- 
stream is transferred from the decoder buffer to the 
demultiplexer 203 one picture at a time in response to 
timing control data located in headers embedded in the 25 
bitstream. The demultiplexer divides the portion of the 
compressed input bitstream relating to each picture into 
its constituent bitstream parameters. The bitstream 
parameters collectively define each picture of the video 
bitstream represented by the compressed input bit- 30 
stream, and include coded, quantized DCT coeff icients, 
motion vectors, quantizing tables, macroblock-type 
information, color data, and header information. The 
header information includes statistical information such 
as the number of bits per block and the number of bits 35 
per picture. The decoder bitstream parameter bus 136 
interconnects the output of the demultiplexer with inputs 
of the other decoding modules, and passes the bit- 
stream parameters from the demultiplexer to the other 
decoding modules. The decoding modules each per- 40 
form a decoding operation using not only the data 
received from one or more preceding decoding modules 
but also the bitstream parameters received via the 
decoder bitstream parameter bus. In addition, the 
decoder bitstream parameter bus is connected to the 4t 
bitstream parameter forwarding module 1 30 to form part 
of the bitstream parameter forwarding path 128. 

The bitstream parameter forwarding module 130 
receives the bitstream parameters from the layered 
decoder 114 via the decoder bitstream parameter bus st 
136. The bitstream parameter forwarding module tem- 
porarily stores those of the bitstream parameters that 
will be used by either or both of the decoded bitstream 
processor 118 and the layered encoder 124 until they 
are needed. The bitstream parameter forwarding mod- a 
ule also feeds the bitstream parameters to an appropri- 
ate channel of the encoder bitstream parameter bus 
140. The encoder bitstream parameter bus connects 
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the bitstream parameters to the encoding modules of 
the layered encoder in which the bitstream parameters 
will later be used to re-encode the modified bitstream 
122. 

The inverse variable length coder 205 receives the 
coded, quantized DCT coefficients from the demulti- 
plexer 203 and reverses the variable length coding 
applied to these coefficients to generate quantized DCT 
coefficients. The quantized DCT coefficients are fed to 
the inverse quantizer 207 and also to the output bus 1 34 
for connection to the output selector 215. In addition, 
the quantized DCT coefficients may be fed as bitstream 
parameters to the bitstream parameter forwarding path 
128 for forwarding to the layered encoder 124. 

The inverse quantizer 207 receives the quantized 
DCT coefficients from the inverse variable length coder 
205, and also receives the quantizing tables from the 
demultiplexer 203 via the decoder bitstream parameter 
bus 136. The inverse quantizer reverses the quantizing 
applied to the quantized DCT coefficients to generate 
blocks of DCT coefficients. The DCT coefficients are fed 
to the inverse DCT module 209, and also to the output 
bus 134 for connection to the output selector 215. In 
addition, the DCT coefficients may be fed as bitstream 
parameters to the bitstream parameter forwarding path 
128 for forwarding to the layered encoder 124. 

The inverse DCT module 209 receives the blocks of 
DCT coefficients from the inverse quantizer 207 and 
applies an inverse discrete cosine transform to the 
blocks of DCT coefficients to generate respective blocks 
of differences. The blocks of differences are fed to the 
motion compensator 21 1 . and also to the output bus 
134 for connection to the output selector 215. In addi- 
tion, the blocks of differences may be fed as bitstream 
parameters to the bitstream parameter forwarding path 
128 for forwarding to the layered encoder 124. 

The motion compensator 211 receives the blocks of 
differences from the inverse DCT module 209 and also 
receives respective reference blocks from a recon- 
structed picture stored in a page of a picture memory 
(not shown) in the spatial bitstream output module 213. 
The picture memory selects the respective reference 
blocks from the reconstructed picture in response to the 
motion vectors for the blocks of differences. The picture 
memory receives the motion vectors from the multi- 
plexer 203 via the decoder bitstream parameter bus 
136. The motion compensator adds each difference in 
the block of differences to a corresponding pixel value in 
the reference block to generate a new pixel value of a 
new reconstructed picture block. The motion compen- 
sate' feeds the reconstructed picture block to the spatial . 
btetream output module 213. The motion compensator 
also feeds the reconstructed picture block onto the out- 
put bus 134 for connection to the output selector 215. 
The successive reconstructed picture blocks fed to the 
output selector constitute a block-format video bitstream 
in which the pictures are in coding order. Coding order 
differs from presentation order, i.e., the order in which 
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the pictures are presented to the viewer. 

When the block of differences is a block of a picture 
that has been encoded as a B-picture, the reference 
block may be selected from two reconstructed pictures 
stored in the picture memory. However, to simplify the 
following description, a reference to selecting a block of 
a reconstructed picture will be taken to encompass 
selecting blocks of more than reconstructed picture 
when the picture is decoded or encoded as a B-picture. 

The spatial bitstream output module 213 receives 
each reconstructed picture block from the motion com- 
pensator 21 1 and stores the reconstructed picture block 
in the picture memory as a block of a reconstructed pic- 
ture being constructed in a page of the picture memory. 
This page is different from the page in which the picture 
that supplied the reference block is stored. When all the 
blocks of a reconstructed picture have been stored in 
the picture memory, the reconstructed picture is availa- 
ble for the spatial bitstream output module to read it out 
onto the output bus 134 for connection the output selec- 
tor 215. The spatial bitstream output moduie reads out 
the reconstructed picture in raster scan format and in 
presentation order as a picture of a conventional video 
bitstream. 

As noted above, the inverse variable-length coder 
205. the inverse quantizer 207, the inverse DCT module 
209, the motion compensator 211 and the spatial bit- 
stream output module 213 operate selectively in 
response to the decoding control data received from the 
processing selection module 120 via the control path 
132. The decoding control data select the decoding 
modules actually used to decode the compressed input 
bitstream. For example, if the user commands the sys- 
tem to perform spatial filtering on the compressed input 
bitstream. and the decoded bitstream processor 1 18 is 
capable of performing such filtering on blocks of DCT 
coefficients, the decoding control data will enable the 
inverse variable-length coder 205 and the inverse quan- 
tizer 207, and will cause the output selector 21 5 to con- 
nect the partially-decoded bitstream from the inverse 
quantizer to the decoded bitstream processor 118 as 
the decoded bitstream 116. The decoding control data 
will also deactivate the decoding modules downstream 
of the decoding module whose output is selected as the 
decoded bitstream 116, i.e., the inverse DCT module 
209, the motion compensator 21 1 and the spatial bit- 
stream output module 213. These decoding modules 
can be deactivated since they are not required to 
decode the compressed input bitstream to the DCT 
coefficient level. This enables computational resources 
that would otherwise be used for these decoding mod- 
ules to be used for other purposes. 

In the layered decoder 1 14. the output selector 215 
is connected to the output bus 134, through which it 
receives the outputs of the decoding modules 201-213. 
The output selector is also connected to the control path 
1 32 through which it receives the decoding control data. 
The output of the output selector is connected to the 



input of the decoded bitstream processor 118. In 
response to the decoding control data, the output selec- 
tor selects the output of one of the decoding modules 
that will serve as the decoded bitstream 116, and feeds 
5 this bitstream to the input of the decoded bitstream 
processor 118. 

It should be noted that when the compressed bit- 
stream processing apparatus 100 accorcfing to the 
invention is embodied in a computer or digital signal 

10 processor, the computer or digital signal processor may 
lack elements such as the bitstream parameter forward- 
ing path 128, the bitstream parameter forwarding mod- 
ule 130, the output bus 134 and the decoder bitstream 
parameter bus 136. Instead, for example, the output 

75 selector 215 may communicate to the decoded bit- 
stream processor 118 the addresses where the data 
output by the last of the decoding modules to decode 
the compressed input bitstream are stored. The 
decoded bitstream processor can then fetch the data 

20 stored at these addresses and perform the decoded bit- 
stream processing on these data Similarly, the bit- 
stream parameter forwarding path 1 28 may 
communicate to the layered encoder 1 24 the addresses 
at which the bitstream parameters derived from the 

25 compressed input bitstream by the layered decoder are 
stored. The appropriate encoding module of the layered 
encoder 124 can then fetch the bitstream parameters 
stored at these addresses to use them in performing its 
encoding processing. It should also be noted that, in 

30 computer-based and DSP-based embodiments, the 
decoding modules may be ephemeral, and the appara- 
tus may lack elements corresponding to those of the 
decoding modules that are not used to decode the com- 
pressed input bitstream 112. 

35 The decoded bitstream processor 118 will be 
described next. The decoded bitstream processor 
receives the decoded bitstream 116 from the layered 
decoder 114 and processes the decoded bitstream to 
generate the modified bitstream 122. The decoded bit- 

40 stream processor can perform any conventional 
processing operation that is not part of the MPEG com- 
pression. These processing operations include color 
correction, scaling, sharpening, noise reduction, etc. In 
the example shown, the processing operation per- 

45 formed by the decoded bitstream processor 1 1 8 on the 
decoded bitstream 1 1 6 is user specific, and is set by the 
user's input to the processing selection module 120. 

The processing operation performed by the 
decoded bitstream processor 1 18 depends on the com- 

so pression state of the decoded bitstream 116. For exam- 
ple, the decoded bitstream processor can perform a 
picture sharpening operation on either the conventional 
video bitstream generated by the spatial bitstream out- 
put module 213 or the DCT coefficients generated by 

55 the inverse quantizer 207. To achieve the desired pic- 
ture sharpening result, the decoded bitstream proces- 
sor will perform a different picture sharpening operation, 
depending on whether the decoded bitstream 116 was 
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generated by the spatial bitstream output module or by 
the inverse quantizer. 

Whether the decoded bitstream 116 processed by 
the decoded bitstream processor 1 1 8 is the result of the 
layered decoder 114 fully decoding or only partially 
decoding the compressed input bitstream 112 depends 
on a trade-off between picture quality and computa- 
tional complexity. For example, when the decoded bit- 
stream processor performs a picture sharpening 
operation on the conventional video bitstream gener- 
ated by the spatial bitstream output module 213 of the 
layered decoder, the results are typically subjectively 
better than when those obtained when the decoded bit- 
stream processor performs a picture sharpening opera- 
tion on the DCT coefficients generated by inverse 
quantizer 207. However, the decoded bitstream proces- 
sor operating on the conventional video bitstream is less 
cost effective because more computational resources 
must be expended to provide the additional decoding 
and encoding modules required for the layered decoder 
to decode the compressed input bitstream to the con- 
ventional video bitstream level and for the layered 
encoder to re-encode the modified video bitstream from 
the conventional video bitstream level. 

In instances in which the compressed bitstream 
processing apparatus 100 is configured to modify the 
bitstream characteristics of the compressed input bit- 
stream 112 without modifying the video signal repre- 
sented by the compressed input bitstream, the 
processing selector 120 configures the decoded bit- 
stream processor 1 1 8 as an identity unit, and the modi- 
fied bitstream 122 is identical to the decoded bitstream 
116. Examples of modifying the bitstream characteris- 
tics of the compressed input bitstream include changing 
the bit rate, changing the reference buffer size, chang- 
ing the quantizing table, etc. 

The layered encoder 124 will be described next 
also with reference to Figures 2 and 3. A key feature of 
the layered encoder lies in that it re-encodes the modi- 
fied bitstream 122 using the bitstream parameters 
derived from the compressed input bitstream by the lay- 
ered decoder 114. Using the bitstream parameters in 
the layered encoder enables the layered encoder to re- 
encode the modified bitstream using fewer computa- 
tional resources than if it generated the bitstream 
parameters anew. For example, re-encoding the modi- 
fied bitstream using the motion vectors and DCT infor- 
mation derived from the compressed input bitstream 
can significantly reduce the computational resources 
required to perform many modifications to the bitstream 
characteristics of the compressed input bitstream. For 
example, such modif ications of the bitstream character- 
istics as bit-rate conversion, decoder buffer size 
change, and picture size scaling can be performed. 
When such modifications of the bitstream characteris- 
tics are performed, the re-encoding can be performed 
using either the original motion vectors derived from the 
compressed input bitstream, or new motion vectors. 



The new motion vectors are generated using the origi- 
nal motion vectors as the starting points for a signifi- 
cantly simpler local search. The decision on whether to 
expand the additional computing resources required to 

5 generate new motion vectors involves a trade-off 
between the quality of the motion vectors and the addi- 
tional computing resources required. When the original 
motion vectors are used to re-encode the modified bit- 
stream after picture scaling, they must also be scaled in 

to proportion to the picture scaling factor. 

The layered encoder 124 can also re-encode the 
modified bitstream 122 using information relating to the 
DCT coefficients of the compressed input bitstream 112 
as bitstream parameters. This provides both computa- 

15 tional expediency and reduces the computational 
resources required to control the encoder buffer and 
output bit rate. For example, the DCT module of the lay- 
ered encoder can use bitstream parameters that indi- 
cate the locations of the non-zero DCT coefficients in 

20 each block in the compressed input bitstream to 
improve the speed at which it computes the forward 
DCT. The DCT module need only compute the DCT 
coefficients for the non-zero locations indicated by the 
bitstream parameters. Furthermore, the quantizer of the 

25 layered encoder can use bitstream parameters that indi- 
cate the bit statistics of the DCT coefficients of the com- 
pressed input bitstream to control the quantizing factor 
"MQUANT." 

The layered encoder 124 generates the com- 
30 pressed output bitstream 126 from the modified bit- 
stream 1 22 output by the decoded bitstream processor 
1 18. In the example shown in Figures 2 and 3, the com- 
pressed output bitstream is an MPEG-compliant video 
bitstream with a new set of user-specific MPEG header 
35 constraints defined by the processing selection module 
120. However, the header information of the com- 
pressed output bitstream may or may not be the same 
as that of the compressed input bitstream 1 1 2 in all lay- 
ers except the data layer. The layers include sequence, 
40 GOP, picture, slice, MB, and any extension header infor- 
mation. 

It is important to understand the principal difference 
between layered encoder 124 and the decoded bit- 
stream processor 118. Unlike the decoded bitstream 

45 processor, the layered encoder is only responsible for 
generating the compressed output bitstream 126 based 
on the user-specified header information and the modi- 
fied bitstream 122 from the decoded bitstream proces- 
sor 118. The layered encoder does not alter the video 

so signal represented by the compressed output bitstream, 
except that the quantizer of the layered encoder may . 
quantize the DCT coefficients differently from the way in 
which they were quantized in the compressed input bit- 
stream. 

55 The layered encoder 124 receives the modified bit- 
stream 1 22 from the decoded bitstream processor 118, 
re-encodes the modified bitstream using bitstream 
parameters derived from the compressed input bit- 
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stream 112 by the layered decoder 114, and supplies 
the resulting compressed output bitstream 126 as the 
output of the compressed bitstream processing appara- 
tus 100. The bitstream parameters are forwarded from 
the layered decoder to the layered encoder via the bit- 
stream parameter forwarding path 128. The layered 
encoder re-encodes the modified bitstream only to the 
extent required by the compression state of the modif ied 
bitstream and the need to generate the compressed 
output bitstream 126 with the bitstream characteristics 
set by the encoder control data. 

The layered encoder 124 operates according to 
encoding control data supplied by the processing selec- 
tion block 120 via the control path 142. The encoding 
control data include header information for the com- 
pressed output bitstream 126. The encoding control 
data depend on processing selections input by the user 
(not shown) into the processing selector 120. 

Like a conventional MPEG encoder, the layered 
encoder 124 includes a number of serially-connected 
encoding modules that operate together to encode the 
modified bitstream 122. Unlike a conventional MPEG 
encoder, the layered encoder can receive the modified 
bitstream in a partially-compressed state, and can use a 
subset of its encoding modules to encode the partially- 
compressed modified bitstream to the extent required to 
provide the compressed output bitstream with bitstream 
characteristics defined by the encoding control data. 

The encoding control data supplied by the process- 
ing selection module 1 20 via the control path 142 deter- 
mine the encoding module at which the layered encoder 
begins encoding. When the modified bitstream 122 is 
uncompressed, i.e., the modified bitstream is a conven- 
tional video bitstream having a pixel value for each pixel 
of each picture, the encoding control data set the lay- 
ered encoder to encode the modified bitstream using all 
the encoding modules. However, when the decoded bit- 
stream 116 processed by the decoded bitstream proc- 
essor 118 was only partially decoded by the layered 
decoder 114, the modified bitstream is partially com- 
pressed, and is not a conventional video bitstream. In 
this case, the encoding control data select a subset of 
the encoding modules for the layered encoder to use to 
complete the encoding of the partially-compressed 
modified bitstream. For example, the decoded bitstream 
processor 118 may perform its processing on a 
decoded bitstream 118 composed of blocks of DCT 
coefficients. In this case, the modified bitstream 122 is 
also partially compressed, and the layered encoder 
encodes this modified bitstream by processing the 
blocks of DCT coefficients of the modified bitstream 
without using the spatial bitstream input module, the 
motion compensator, the DCT module, and the local 
decoder. 

The encoding modules that constitute the layered 
encoder 124 are based on the corresponding modules 
of a conventional encoder. It is assumed that persons of 
ordinary skill in this art are familiar with the structure 



and operation of such conventional encoding modules, 
so the encoding modules will be described only briefly. 
The encoding modules constituting the layered encoder 
include the spatial bitstream input module 313, the 
- motion compensator 31 1 . the discrete cosine transform 
(DCT) module 309, the quantizer 307, the variable- 
length coder 305. the multiplexer 303, the encoder 
buffer 301 and the local decoder 317. 

In addition to the above-mentioned encoding mod- 
w ules, the layered encoder 124 includes the input selec- 
tor 315, the input bus 138 and the encoder bitstream 
parameter bus 140. The input selector has an input con- 
nected to the output of the decoded bitstream processor 
118 through which it receives the modified bitstream 
is \2Z. The input selector has an output connected to the 
input bus 138. The input bus is in turn connected to the 
input of each of the spatial bitstream input module 313, 
the motion compensator 31 1 , the DCT module 309, the 
quantizer 307, the variable-length coder 305. and the 
20 multiplexer 303. The input selector has a control input 
connected to the control path 142 through which it 
receives the encoding control data. In response to the 
encoding control data, the input selector determines 
which of the spatial bitstream input module 313, the 
25 motion compensator 311. the DCT module 309, the 
quantizer 307. and the variable-length coder 305 is the 
first of the encoding modules that will process the mod- 
ified bitstream. The input selector then connects the 
input of the first-to-process encoding module to the out- 
30 put of the decoded bitstream processor 118 via the 
input bus 138. For example, when the modified beam 
1 22 from the decoded bitstream processor 1 1 8 is a con- 
ventional video bitstream in raster scan format and 
presentation order, the input selector 315 connects the 
35 output of the decoded bitstream processor 1 18 to the 
input of the spatial bitstream input module 313. The 
spatial bitstream input module can then convert the 
raster-scan and presentation-order format of the modi- 
fied bitstream into macroblocks and blocks in coding 
40 order for processing by the remaining encoding mod- 
ules of the layered encoder. 

The encoder bitstream parameter bus 140 has 
inputs connected to the output of the bitstream parame- 
ter forwarding module 130 and outputs connected to an 
45 additional input on each of the encoding modules 303- 
313. The additional inputs connected to the encoder bit- 
stream parameter bus enable the encoding modules to 
receive the bitstream parameters derived from the com- 
pressed input bitstream by the layered decoder 114 and 
so forwarded via the bitstream parameter forwarding path 
128. 

In the layered encoder 124. the spatial input bit- 
stream module 313 receives a conventional video bit- 
stream in raster-scan format and presentation order. 
55 and converts the video bitstream into blocks and mac- 
roblocks in coding order suitable for processing by the 
remaining encoding modules of the layered encoder. 
The spatial input bitstream module feeds the resulting 
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block-formal video bitstream to the motion compensator 
311. 

The motion compensator 31 1 determines a motion 
vector for each macroblock of each picture that will be 
encoded as a P-picture or a B-picture. These pictures 
are the pictures that will not be independently encoded 
as l-pictures. Conventionally, the motion vector for each 
macroblock is determined by performing a block match- 
ing operation between the macroblock and reference 
blocks selected from one or more reference pictures. 
The reference pictures are usually reconstructed pic- 
tures obtained as a result of the local decoder 317 
decoding the bitstream output by the quantizer 307. The 
reconstructed pictures are stored in a multi-page picture 
memory that forms part of the local decoder. 

In a conventional encoder, the block matching oper- 
ation to determine the motion vectors demands the 
most computational resources of any of the operations 
performed by the encoder. However, the computational 
requirements for determining the motion vectors in the 
compressed bitstream processing apparatus 100 are 
substantially reduced or even eliminated by re-using the 
motion vectors derived from the compressed input bit- 
stream 112 by the layered decoder 114 and forwarded 
to the layered encoder via the bitstream parameter for- 
warding path 128. 

The motion compensator 311 additionally proc- 
esses the blocks of the pictures that will be encoded as 
P-pictures and B-pictures to generate difference blocks. 
The blocks subject to this processing will be called pic- 
ture blocks to distinguish them from the reference 
blocks and the difference blocks. The difference block 
for each picture block is obtained by a sitotraction oper- 
ation performed between the picture block and a refer- 
ence block. The reference block is the block of the 
reference picture indicated by the motion vector of the 
picture block. 

The motion compensator 311 also passes encod- 
ing blocks to the DCT module 309. Encoding blocks are 
picture blocks if the picture is to be encoded as an (-pic- 
ture, and are the difference blocks generated as just 
described if the picture is to be encoded as a P-picture 
or a B-picture. 

The DCT module 309 receives the encoding blocks 
from the motion compensator 311 and orthogonally 
transforms each encoding block from the spatial domain 
to the frequency domain using a discrete cosine trans- 
form. The DCT module passes the resulting blocks of 
DCT coefficients to the quantizer 307. 

The quantizer 307 receives the blocks of DCT coef- 
ficients from the DCT module 309 and quantizes each 
of the DCT coefficients in each block The quantizing 
step-size is controlled by a buffer occupancy control sig- 
nal (not shown) generated by the encoder buffer 301. 
The quantizing step size is controlled to ensure that the 
encoder buffer and a hypothetical standard video 
decoder buffer neither overflow nor underflow. The 
quantize- passes the resulting Mocks of quantized DCT 



coefficients to the variable-length coder 305. 

The variable-length coder 305 receives the blocks 
of quantized DCT coefficients from the quantizer 307 
and applies variable-length coding to the quantized 
5 DCT coefficients in each block. The "variable-length 
coder passes the resulting blocks of coded quantized 
DCT coefficients to the multiplexer 303 and to the focal 
decoder 317. 

The multiplexer 303 receives the blocks of coded 
10 quantized DCT coefficients from the variable-length 
coder 305. and additionally receives parameters such 
as motion vectors from the motion compensator 311, 
quantizing step size information from the quantizer 307, 
and run-length information from the variable-length 
is coder 305, and combines these parameters to generate 
a picture-layer video bitstream. The multiplexer passes 
the resulting video bitstream picture-by-picture to the 
output buffer 301. 

The output buffer 301 receives the video bitstream 
20 from the multiplexer 303 and generates the compressed 
output bitstream 126, a constant bit-rate bit stream. An 
additional multiplexer (not shown) may interleave the 
compressed output bitstream with other bitstreams, 
such as a compressed audio bitstream, and with adcfi- 
25 tional header information, to generate a composite out- 
put bitstream. 

The spatial bitstream input module 313, the motion 
compensator 311. the DCT module 309. the quantizer 
307 and the variable-length coder 305 operate selec- 
30 tively in response to the encoding control data received 
from the processing selection module 120 via the con- 
trol path 142. The encoding control data determine 
whether each of the above modules will actually oper- 
ate. For example, if the user desires to perform spatial 
35 filtering on the compressed input bitstream, and the 
decoded bitstream processor is capable of performing 
such filtering on DCT coefficients, the encoding control 
data will enable the quantizer 307 and the variable- 
length coder 305 and will cause the input selector 31 5 to 
40 connect the partially-compressed modified bitstream 
122 from the decoded bitstream processor 118 to the 
input of the quantizer 307. The encoding control data 
will also deactivate the encoding modules upstream of 
the encoding module that receives the modified bit- 
45 stream, i.e., the DCT module 309, the motion compen- 
sator 311. the spatial bitstream input module 313 and 
the local decoder 317. Deactivating the unused encod- 
ing modules frees computational resources that would 
otherwise be used to provide these modules to be used 
so for other purposes. 

It should be noted that when the compressed bit- . 
stream processing apparatus according to the invention 
is embodied in a computer or digital signal processor, 
the computer or digital signal processor may lack such 
55 elements as the input bus 138 and the encoder bit- 
stream parameter bus 140. In this case, the input selec- 
tor 315 may communicate this addresses at which the 
decoded bitstream processor 118 stored the modified 
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bitstream 122 to the first of the encoding modules to 
process the modified bitstream. This encoding module 
will then fetch the modified bitstream from these 
addresses. Similarly, the bitstream parameter forward- 
ing module 130 may communicate to the layered 
encoder 124 the addresses at which the bitstream 
parameters derived from the compressed input bit- 
stream 1 1 2 by the layered decoder 1 1 4 are stored. The 
layered encoder can then fetch the bitstream parame- 
ters stored at these addresses to use them in connec- 
tion with encoding the mocfified bitstream. It should also 
be noted that, in computer-based and DSP-based 
embodiments, the encoding modules may be ephem- 
eral, and the apparatus will lack elements correspond- 
ing to the encoding modules that are not used to encode 
the modified bitstream 122. 

Some of the ways in which the bitstream parame- 
ters derived from the compressed input bitstream 112 
by the layered decoder 114 can be used will now be 
described. In a first example, the bitstream parameters 
derived from the compressed input bitstream are for- 
warded to the decoded bitstream processor 118 where 
they are used to simplify the processing performed by 
the decoded bitstream processor. The bitstream param- 
eters are color data indicating the color characteristics 
of the video signal represented by the compressed input 
bitstream. Such color data are included in the picture 
headers derived from the compressed input bitstream 
by the demultiplexer 203. The demultiplexer forwards 
the color data to the decoded bitstream processor 118 
via the bitstream parameter forwarding path 128. The 
decoded bitstream processor then uses the color data 
to determine whether it needs to apply color correction 
to the decoded bitstream 116. rf the decoded bitstream 
processor determines that color correction is needed, it 
may additionally use the color data to design an appro- 
priate color calibration scheme. 

The layered decoder 1 1 4 decodes the compressed 
input bitstream 1 1 2 to generate the decoded bitstream 
116 in a compression state suitable for processing by 
the decoded bitstream processor 118. The decoded bit- 
stream processor then applies color correction to the 
decoded bitstream using the color calibration scheme it 
has derived from the color data derived from the com- 
pressed input bitstream. The layered encoder 124 then 
re-encodes the modified bitstream 122 generated by 
the decoded bitstream processor 118 to generate the 
compressed output bitstream 126. The video signal rep- 
resented by the compressed output bitstream is color 
corrected relative to that represented by the com- 
pressed input bitstream. Additional bitstream parame- 
ters derived from the compressed input bitstream 112 
may be used by the layered encoder to simplify the 
encoding processing required to re-encode the modified 
bitstream 122, as win be described below. 

As noted above, in conventional MPEG compres- 
sion, the motion estimation operation of the encoding 
process requires the most computational resources. 



Motion estimation is conventionally performed using a 
motion compensator module located in the encoder in a 
position corresponding to that of the motion compensa- 
tor 31 1 in the layered encoder 1 24. Consequently, elim- 

s mating or substantially simplifying the motion estimation 
operation can achieve substantial savings in the compu- 
tational resources required by the layered encoder 124. 

The motion compensator 311 can be completely 
disabled if the decoded bitstream processor 118 can 

w perform the processing operations selected by the user 
using the DCT coefficients derived from the com- 
pressed input bitstream 112 by the layered decoder 
114. This provides a substantial reduction in the compu- 
tational resources demanded by the layered encoder. 

is However, if the processing operation performed by the 
decoded bitstream processor 118 requires that the lay- 
ered decoder fully decode the compressed input bit- 
stream to provide a conventional video bitstream as the 
decoded bitstream 116, then the modified bitstream 122 

20 is a conventional video bitstream that has to be fully re- 
encoded. 

The compressed bitstream processing apparatus 
100 provides a number of ways of reducing the compu- 
tational resources required to re-encode the modified 
25 bitstream when the modified bitstream is a conventional 
video bitstream. The bitstream parameter forwarding 
path 128 forwards the motion vectors derived from the 
compressed input bitstream 1 12 to the layered encoder 
124. The layered encoder can then use these motion 
30 vectors as the motion vectors in the motion compensa- 
tion operations performed by the motion compensator 
31 1 . This provides a significant reduction in the compu- 
tational resources required by the layered encoder 124, 
since no motion estimation operations are performed. 

35 Re-using the motion vectors derived from the com- 
pressed input bitstream constrains the performance of 
the motion estimation in the layered encoder to be no 
better than that originally used to encode the com- 
pressed input bitstream. However, this is not usually a 

40 significant drawback since most hardware or software 
encoders employ an effective although efficient motion 
estimation algorithm. 

Simply re-using the motion vectors derived from the 
compressed input bitstream 1 1 2 will produce an accept- 

45 able motion estimation accuracy, but a better motion 
estimation accuracy is obtained by using the motion 
vectors derived from the compressed input bitstream as 
full-pixel motion vectors, and performing an additional 
half-pixel search relative to the full-pixel motion vectors 

so to produce the final motion estimation. This alternative 
demands more computational resources than re-using 
the motion vectors derived from the original com- 
pressed input bitstream. However, it produces a better 
picture quality and a better compression efficiency. 

55 Moreover, since the scope of the motion estimation 
search is considerably less than that used in conven- 
tional motion estimation, the additional computing 
resources required are substantially less than those 
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required for conventional motion estimation. 

An example of the structure of the motion compen- 
sator 311 that performs local motion estimation is 
shown in Figure 4. The motion compensator includes 
the local motion essimator 441 and the subtracter 443. 
The local motion estimator has inputs connected to the 
outputs of the spatial bitstream input module 313, the 
encoder bitstream parameter bus 140, and the local 
decoder 317. The local motion estimator has outputs 
connected to the reconstructed picture memory 445 in 
the local decoder, the encoder bitstream parameter bus 
and both inputs of the subtracter 443. 

The local motion estimator 441 receives successive 
picture blocks from the spatial bitstream input module 
313 and performs a motion estimation operation on 
each of them. The picture block thai is the subject of the 
current motion estimation operation will be called the 
current picture block. The local motion estimator also 
receives the motion vector for the current picture block 
from the encoder bitstream parameter bus 140. This 
motion vector was derived from the compressed input 
bitstream 112 as a bitstream parameter, and was for- 
warded via the bitstream parameter forwarding path 1 28 
(Figure 2). The local motion estimator derives an 
address from the forwarded motion vector for the cur- 
rent picture block, and supplies this address to the pic- 
ture memory 445 of the local decoder 31 7. In response, 
the picture memory feeds a super reference block back 
to the local motion estimator. The super reference block 
is a block of a reconstructed picture stored at the loca- 
tion in the picture memory designated by the address, 
and is two pixels larger than a conventional picture block 
in both the horizontal and vertical directions. The larger 
size of the super reference block enables matching 
blocks having a half-pixel precision to be generated 
from the super reference block. 

The local motion estimator 441 performs interpola- 
tion operations between adjacent pixel values in the 
super reference block to generate a set of eight 8x8 
pixel matching blocks. Each matching block is displaced 
relative to the center of the super matching block in a 
different one of the directions N, NE, E, SE, S, SW. W, 
NW by a distance equivalent to one half of the distance 
between adjacent pixels in the super reference block. 
The local motion estimator also generates a ninth 
matching block by selecting an 8 x 8 pixel picture block- 
sized subset centered in the 10 x 10 pixel super refer- 
ence block. 

The local motion estimator 441 then performs a 
block matching operation between the current picture 
block and the nine matching blocks to determine which 
matching block most closely matches the current pic- 
ture block. This matching block will be called the closest 
matching block. The local motion estimator then adds 
the displacement of the closest matching block relative 
to the forwarded motion vector to the forwarded motion 
vector to generate a new motion vector with half-pixel 
precision for the current picture block. The local motion 



estimator also adopts the closest matching block as the 
reference block for the current picture block. 

The local motion estimator 441 feeds the new 
motion vector with half-pixel precision to the encoder 

5 bitstream parameter bus 140 for forwarding to the 
encoder buffer 301 , and feeds the current picture block 
and its reference block to the inputs of the subtracter 
443. The subtracter 443 performs a pixel-by-pixel sub- 
traction of the reference block from the current picture 

10 block, and feeds the resulting difference block to the 
DCT module 409. 

Bitstream parameters derived from the compressed 
input bitstream 112 may also be used to reduce the 
computational requirements of the bit rate control per- 

is formed by the layered encoder 124. Bit rate control 
plays an important rdle in achieving good video quality 
and helps ensure that the compressed output bitstream 
1 26 can be decoded in real time by a practical decoder. 
Bit rate control controls the number of bits allocated to 

20 encode each picture. The number of bits allocated to 
encode each picture must not only quantize all the DCT 
coefficients in the picture with a sufficient number of 
quantizing levels to achieve an acceptable presentation 
quality, but must also satisfy the buffer overflow and 

25 underf tow constraints defined by the MPEG standard. 

The compressed input bitstream 112 contains bit- 
stream parameters indicating the characteristics of 
each macroblock. The layered decoder 114 can derive 
such bitstream parameters from the compressed input 

30 bitstream, and pass them to the layered encoder 124 
where they are re-used to assist in performing the adap- 
tive bit allocation. Examples of two different approaches 
to re-allocating bits to the DCT coefficients of each mac- 
roblock will be described with reference to Figures 3, 5 

35 and 6. 

In the first approach, illustrated in Figure 5, the lay- 
ered decoder 114 derives the quantizing step sii* 
' n MQUAN*r[*] from tne compressed input bitstream 112. 
The quantizing step size /n MQUANT [/c] is the quantizing 

40 step size of the current macroblock k in the compressed 
input bitstream. The layered decoder passes the quan- 
tizing step size to the layered encoder as a bitstream 
parameter. The quantizing step size is also passed to 
the inverse quantizer 207, which performs an inverse 

45 quantizing operation on the DCT coefficients in the cur- 
rent macroblock k using this quantizing step size. Then, 
in the layered encoder 124, the quantizer 307 calculates 
a new quantizing step size MQUANT[/c] for quantizing 
the DCT coefficients of the current macroblock k and re- 

50 quantizes the DCT coefficients of the current macrob- 
lock k with the new quantizing step size. The quantizer . 
calculates the new quantizing step size MQUANTI/c] 
using the quantizing step size for the current macrob- 
lock k forwarded from the compressed input bitstream. 

55 In the second approach, illustrated in Figure 6. the 
quantized DCT coefficients for the current macroblock k 
are derived from the compressed input bitstream by the 
inverse variable-length coder 205 and forwarded 
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through the bitetream parameter forwarding path 128 
from the layered decoder 1 1 4 to the quantizer 307 of the 
layered encoder 124. Also, the bitstream parameter 
mb inco0 fj[k] indicating the number of non-zero DCT 
coefficients in the current macroblock is derived from 
the compressed input bitstream and forwarded from the 
layered decoder to the quantizer in the layered encoder- 
Using the bitstream parameter mb incoe ff[k] t the quan- 
tizer then performs an operation that nulls out selected 
ones of the non-zero quantized DCT coefficients such 
that the compressed output bitstream meets the above- 
mentioned buffer constrains. 

Nulling out selected non-zero quantized DCT coef- 
ficients in the second approach demands fewer compu- 
tational resources than the first approach because 
inverse quantizing and re-quarrtizing are not performed. 
However, the first approach will usually provide a better 
presentation quality. 

The computational resources required by the first 
approach are reduced by using bitstream parameters 
derived from the compressed input bitstream to enable 
the adaptive bit allocation of the compressed input bit- 
stream to assist in controlling the bit rate of the com- 
pressed output bitstream. The processing performed in 
the f irst approach is illustrated in Figure 5. In the follow- 
ing description, the block of DCT coefficients that is the 
subject of the current quantizing operation will be called 
the current block, and the picture of which the current 
block forms part will be called the current picture. 

At the start of encoding, steps S1 and S2 are per- 30 
formed. In step S1 , the layered decoder 1 14 derives the 
bitstream parameter R in from the compressed input bit- 
stream 1 1 2 and forwards this bitstream parameter to the 
quantizer 307 via the bitstream parameter forwarding 
path 128. The bitstream parameter R in indicates the bit 35 
rate of the compressed input bitstream. In step S2, the 
quantizer 307 in the layered encoder 124 receives the 
bitstream parameter R im from the encoder bitstream 
parameter bus 140, and receives the parameter R out 
from the processing selection module 120. The quan- 40 
tizer then determines the ratio bits rst joo by dividing R out 
by 

In step S3, the inverse variable-length coder 203 
derives the bitstream parameter /n MQUAN -rlfc] from each 
macroblock in the compressed input bit stream 112. 45 
The bitstream parameter in MQUAJN j[k] of the current 
macroblock k of the compressed input bitstream indi- 
cates the quantizing step size that was used to quantize 
the DCT coefficients of the current macroblock k. The 
inverse variable-length coder forwards the bitstream so 
parameter /n MQUANT [fc] to the quantizer 307 in the lay- 
ered encoder 124. 

In step S4, the inverse variable-length coder 203 
derives the bitstream parameter inbits[k] for the current 
macroblock k by counting the number of bits in the com- ss 
pressed input bitstream used to represent the DCT 
coefficients of the current picture prior to the DCT coef- 
ficients of the current macroblock k. The inverse varia- 



ble-length coder forwards the bitstream parameter 
inbits[k] to the quantizer 307 in the layered encoder 
124. 

In steps S5-S8. the quantizer,307 in the layered 
5 encoder 124 uses the bitstream parameters forwarded 
by the layered decoder 1 14 to quantize the DCT coeffi- 
cients in the current macroblock with a quantizing step 
size that enables the bit rate of the compressed output 
bitstream 126 to comply with the above-mentioned 
10 buffer constraints. The variable-length coder 305 
includes a counter that counts outbits[k\ t the number of 
bits of the compressed output bitstream used to repre- 
sent the DCT coefficients of the current picture prior to 
the DCT coefficients of the current macroblock k. The 
15 variable-length coder feeds each value of outbits[k] 
back to the quantizer. Finally, the quantizer receives 
encoding control data via the control path 142. 

In step S5, the quantizer determines the ratio 
bitSratiolk] by determining the ratio between outbits[k] 
20 and inbits[k], i.e.: 

bits ratjo [k] = outbits[k]\nbits[k] 

In step S6. the quantizer 307 determines the con- 
25 version factor f[k] for the current macroblock k. The 
quantizer calculates the conversion factor from the 
ratios bits^ tio lk] and bits rati(>0 using the following equa- 
tion: 

xri ^ 2 x bits mtk> [k\+ d/te 

ratio O 

rw = bits^lk] + 2 x Ms rafo0 

where 0.7 <> [k] < 1.3. 

In step S7, the quantizer 307 calculates the value of 
MQUANT[fr] for the current macroblock k. The value of 
MQUANT[fc] is the quantizing step size that the quan- 
tizer will use to quantize the DCT coefficients of the cur- 
rent macroblock k . The quantizer calculates the value of 
MQUANT[/c] for the current macroblock k by multiplying 
the value of the bitstream parameter /nMQUANTM by the 
conversion factor f[k], i.e. : 

MQUANT[/c] = in MQUANT lk] x f[k). 

In step S8, the quantizer 307 quantizes all the non- 
zero DCT coefficients in the current macroblock k using 
the quantizing step size MQUANT[/c]. 

In step S9, the quantizer determines whether all the 
macroblocks in the current bitstream have been quan- 
tized. If so. proceiling proceeds to step S10, where it 
ends. Otherwise, processing returns to step S3 and the 
process of calculating the ratio b/Ys^^I/f], the conver- 
sion factor f[k] and the quantizing step size MQUANT[/c] 
for the next macroblock k , and quantizing the DCT coef- 
ficients of the next macroblock using the quantizing step 
size MQUANTI/c] is repeated until the picture is com- 
pletely coded, and until the entire compressed input bit- 
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stream has been processed. 

The computational resources required for the sec- 
ond approach are also reduced by using bitstream 
parameters derived from the compressed input bit- 
stream to enable the adaptive bit allocation of the com- s 
pressed input bitstream to assist in controlling the bit 
rate of the compressed output bitstream. The process- 
ing performed in the second approach is illustrated in 
Figure 6. Steps S1 , S2, S4-S6, and S9 are the same as 
in the first approach described above with reference to 10 
Figure 5, and will not be described again here. 

In step S3, the layered decoder 1 1 4 derives the bit- 
stream parameter mb incoef Ak] each macroblock of the 
compressed input bitstream 112 and forwards this bit- 
stream parameter to the quantizer 307 in the layered is 
encoder 124 via the bitstream parameter forwarding 
path 128. The bitstream parameter mb inco0 fj[k] indi- 
cates the number of non-zero DCT coefficients in the 
current macroblock k of the compressed input bit- 
stream. 20 

In step S5A, the layered decoder 114 forwards the 
quantized DCT coefficients derived from the com- 
pressed input bitstream 112 by the inverse variable- 
length coder 205 to the quantizer 307 via the bitstream 
parameter forwarding path 1 28. 25 

In step S7. the quantizer 307 in the layered encoder 
124 uses the bitstream parameter mb incoef &k] to deter- 
mine the number of the quantized DCT coefficients 
received from the layered decoder that will be set to 
zero to control the bit rate of the compressed output bit- so 
stream. The quantizer calculates the value of 
mb outcoef /lk\ for the current macroblock k. The value of 
mb outcoe dik\ is the number of DCT coefficients in the 
macroblock k that may have a value other than zero. To 
calculate the value of mb outcoef J[k] for the current mac- 35 
roblock k, the quantizer multiplies the value of the bit- 
stream parameter mb incoe dk] by the value of the 
conversion factor f[k] calculated in step S6. i.e., 

MbouteoeffW = mb^lk] x f[kl 40 

In step S8, the quantizer 307 receives all the quan- 
tized DCT coefficients of the current macroblock k from 
the inverse variable-length coder 205, and sets to zero 
all but the number of them indicated by the value of 45 

In the embodiment just described, the compressed 
input signal and the compressed output signal were 
both MPEG-compliant bitstr earns. An example of a sec- 
ond embodiment 700 of a compressed bitstream sc 
processing apparatus according to the invention will 
now be described with reference to Figures 7A and 7B. 
This embodiment operates on a compressed input bit- 
stream in any compression state, generates the com- 
pressed output bitstream in any compression state, end st 
allows multiple processors operating on brtstreams hav- 
ing different compression states to be concatenated. In 
the example to be described, the compressed bitstream 



processing apparatus processes a compressed input 
bitstream representing a video signal. 

The compressed bitstream processing apparatus 
700 is composed of four basic units, the first compres- 
sion state translation unit 714. the final" compression 
state translation unit 724, the processor 71 8A, and the 
bitstream parameter forwarding path 728 that includes 
the bitstream parameter forwarding module 730. If the 
compressed bitstream processing apparatus is 
designed to allow a user to select or define the process- 
ing operation performed, the apparatus additionally 
includes the processing selection module 720 that 
receives instructions from the user and controls the 
processing performed by the compression state transla- 
tion units and the processor. Otherwise, the processing 
selection module controls the processing performed by 
the compression state translation units and the proces- 
sor so that the compressed bitstream processing appa- 
ratus performs a predetermined processing operation 
on the compressed input bitstream. 

The example shown in figure 7A includes two 
optional additional units, the compression state transla- 
tion unit 71 4B and the second processor 71 8B. These 
additional units provide . the compressed bitstream 
apparatus 700 with the capability to process the infor- 
mation signal represented by the compressed input bit- 
stream in two different ways, a first way performed by 
the first processor 71 8A and a second way performed 
by the processor 718B. The compression state transla- 
tion unit 71 4B precedes the optional second processor 
71 8B. and the two units are interposed between the first 
processor 71 8A and the final compression state transla- 
tion unit 724. The compression state translation unit 
714B and the second processor are also connected to 
the bitstream parameter forwarding path 728 and are 
controlled by the processing selection unit 720. 

Preceding the second processor 71 8B with the 
compression state translation unit 71 4B enables the 
second processor to operate on a bitstream having a 
different compression state from the bitstream output by 
the first processor 71 8A. Optionally, more than one 
additional processor, each preceded by a compression 
state translation unit, may be interposed between the 
first processor 71 8A and the final compression state 
translation unit 724. 

In the example shown in figure 7 A, the video signal 
represented by the compressed input bitstream 712 is 
subject to two different processing operations, one per- 
formed by the first processor 71 8A and the other per- 
formed by the second processor 71 8B. The first 
processor is preceded by the compression state trans- . 
lation unit 714A. which changes the compression state 
of the compressed input bitstream to generate the out- 
put bitstream 71 6A in the compression state required by 
the first processor. The first compression state transla- 
tion unit enables the compressed input bitstream 71 2 to 
have a compression state that differs from that required 
by the first processor. 
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The first processor 71 8A processes the output bit- 
stream 71 6A generated by the compression state trans- 
lation unit 71 4A to generate the modified bitstream 
722 A. The first processor modifies the video signal rep- 
resented by the output bitstream 716A f but does not 
change the compression state of the output bitstream. 
Consequently, the compression state of the modified 
bitstream 722A is the same as that of the output bit- 
stream 716A. Optionally, the first processor 714A may 
be composed of multiple concatenated processors, all 
of which operate on bitstreams in the same compres- 
sion state. 

The second processor 71 8B is preceded by the 
compression state translation unit 71 4B, which changes 
the compression state of the modified bitstream 722A to 
generate the output bitstream 716B in the compression 
state required by the second processor. The compres- 
sion state translation unit 71 4B enables the second 
processor to be concatenated with the first processor 
71 8A. even though the processors operate on input bit- 
streams having differing compression states. 

The second processor 71 8B processes the output 
bitstream 71 6B generated by the compression state 
translation unit 714B to generate the modified bitstream 
722B. The second processor modifies the video signal 
represented by the output bitstream 71 6B, but does not 
change the compression state of the output bitstream. 
Consequently, the compression state of the modified 
bitstream 722B is the same as that of the output bit- 
stream 71 6B. Optionally, the second processor may 
also be composed of multiple concatenated processors, 
all of which operate on bitstreams in the same compres- 
sion state. 

The second processor 71 8B is followed by the final 
compression state translation unit 724, which changes 
the compression state of the modified bitstream 722B to 
generate the compressed output signal 726. This ena- 
bles the compressed bitstream processing apparatus 
700 to deliver the compressed output signal in the 
desired compression state. The final compression state 
translation unit may also change the bitstream charac- 
teristics. e.g.. bit rate, buffer size, etc., of the modified 
bitstream to generate the compressed output bitstream 
with the desired bitstream characteristics. The com- 
pression state and bitstream characteristics of the com- 
pressed output signal may be the same as, or different 
from, that of the compressed input bitstream 712. 

Figure 78 shows details of the compression state 
translation unit 714 having the same structure and oper- 
ation as the compression state translation units 71 4A, 
71 4B and 724 of the embodiment shown in Figure 7 A. 
The compression state translation unit 71 4 receives the 
input bitstream 712 in a first compression state, selec- 
tively changes the compression state of the input bit- 
stream to generate the output bitstream 716 having a 
different compression state from the input bitstream. 
The compression state of the output bitstream may be 
greater than or less than that of the input bitstream, i.e., 



the compression state translation unit 714 may operate 
in an encoding mode or a decoding mode, depending 
on the control data received via the control path 732. 
The compression state translation unit 714 is com- 
5 posed of three main modules, the input interface 719, 
the output interface 721. and the layered encod- 
ing/decoding modules 723. The layered encod- 
ing/decoding modules are composed of an 
encoding/decoding module corresponding to each corn- 
to pression state of the compressed input bitstream. In the 
example shown, the layered encocfing modules are the 
variable length coding/decoding module 705, the quan- 
tizing/inverse quantizing module 707, the orthogonal 
transform module 709 and the motion compensa- 
is tion/spatial processing module 71 1 . 

The input interlace module 719 includes buffering 
and demultiplexing elements (not shown) that derive bit- 
stream parameters from the input bitstream 712. The 
input interface module selectively feeds the bitstream 
20 parameters to those of the layered encoding/decoding 
modules 723 that require them via the input bus 738. 
The input bus enables the input interface module to feed 
bitstream parameters to any of the encoding/decoding 
modules. The control data received via the control path 
25 732 determine which of the encoding/decoding mod- 
ules receive the bitstream parameters from the input 
bus. 

The input interface module 719 also feeds the bit- 
stream parameters to the bitstream parameter forward- 
30 ing bus 736 for forwarding to the processors 71 8A and 
71 8B and to other compression state translation units 
via the bitstream parameter forwarding path 728. The 
control data received via the control path 732 determine 
which of the bitstream parameters are fed to the bit- 
35 stream parameter forwarding bus 736. 

The bitstreams generated by the layered encod- 
ing/decoding modules 723 are fed to the data output 
bus 725. These bitstreams are composed of such data 
elements as pixel values, motion vectors, coded, quan- 
40 tized DCT coefficients, etc. The data output bus trans- 
fers these bitstreams to an encoding/decoding module 
of a higher or lower layer, and also to the output inter- 
face module 721. 

The output interface module 721 includes multiplex- 
es ing and buffering elements (not shown) that construct 
the output bitstream 716 from the bitstreams generated 
by one or more of the layered encoding/decoding mod- 
ules 723. The output interface module can select the bit- 
stream output by any of the encoding/decoding 
so modules for inclusion in the output bitstream. The out- 
put interface module selects the encoding/decoding 
modules from to receive a bitstream for inclusion in the 
output bitstream in response to the control data 
received via the control path 732. 
55 The layered encoding/decoding modules 723 are 
all dual-function modules. Each of the modules is capa- 
ble of operating in an encoding mode or in a decoding 
mode, depending on the control data received via the 
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control path 732. For example, when the control data 
set the orthogonal transform module 709 to its encoding 
mode, this module performs both forward and reverse 
discrete cosine transforms, the inverse transforms 
being required to provide part of the local decoding 5 
function of an encoder. When the control data set the 
orthogonal transform module to its decoding mode, this 
module only performs inverse discrete cosine trans- 
forms. 

The control data received via the control path 732 10 
also select the encoding/decoding modules required to 
provide the compression state translation unit with its 
desired compression state translation characteristic. 
The control data may select all of the encoding/decod- 
ing modules, or a subset of the encoding/decoding 15 
modules, depending on the compression states of the 
input and output bitstreams. 

In a computer- or DSP-based compressed bit- 
stream processing apparatus, the control data received 
via the control path 732 also determine the allocation of 20 
computational resources to each of the layered encod- 
ing/decoding modules 723. The compression state 
translation unit performs a partial encoding or decoding 
operation using a subset of the layered encod- 
ing/decoding modules. Normally, in this case, no com- 25 
putational resources are allocated to the remaining, 
unused encoding/decoding modules. 

Each of the encoding/decoding modules 723 oper- 
ates on a bitstream output by either the input interface 
module 71 9 or the encoding/decoding module in at least 30 
one higher or lower processing layer. Each of the 
encoding/decoding modules outputs a bitstream to the 
output interface module 721 or to the encoding/decod- 
ing module in at least one high©- or lower processing 
layer. The bitstream routing among the encod- 35 
ing/decoding modules and between the input and out- 
put interface modules 71 9 and 721 and the 
encoding/decoding modules is determined by the con- 
trol data received via the control path 732. Whether a 
bitstream is output to a encoding/decoding module in a 40 
higher layer or a lower layer depends on whether the 
compression state translation unit is operating in its 
encode mode or its decode mode, respectively. 

In addition, the layered encoding/decoding modules 
723 may operate using bitstream parameters received 45 
via the bitstream parameter forwarding path 728, and 
the layered encoding/decoding modules may addition- 
ally or alternatively derive bitstream parameters from 
the compressed input bitstream and forward such bit- 
stream parameters to the other compression state so 
translation units and the processors via the bitstream 
parameter forwarding path 728. 

A first practical example of a compression state 
translation unit will now be described. In this example, 
the compression state translation unit is the first com- 5£ 
pression state translation unit 71 4A of a compressed 
bitstream processing apparatus in which the first proc- 
essor 71 8A operates on a partially-compressed bit- 



stream composed of DCT coefficients. In this example, 
the input bitstream 712 of the first compression state 
translation unit is an MPEG bitstream. and the first com- 
pression state translation unit operates to decode the 
MPEG bitstream to the compression state required by 
the first processor. 

Control data received via the control path 732 
cause the compression state translation unit 71 4A to 
operate in the following way: the variable-length cod- 
ing/decoding module 705 operates in its variable-length 
decoding mode, the quantizing/inverse quantizing mod- 
ule 707 operates in its inverse quantizing mode and the 
remaining encoding/decoding modules are disabled. No 
computational resources are allocated to the disabled 
encoding/decoding modules. 

The input interface module 719 derives coded, 
quantized DCT coefficients, quantizing tables, motion 
vectors, picture headers and other bitstream parame- 
ters from the input bitstream 712. Using the input bus 
738. the input interface module feeds the DCT coeffi- 
cients to the variable-length decoding module 705, the 
quantizing tables to the inverse quantizing module 707, 
and the motion vectors and picture headers to the out- 
put interface module 721, Using the bitstream parame- 
ter forwarding bus 736, the input interface module also 
feeds the quantizing tables, motion vectors, picture 
headers and other bitstream parameters derived from 
the input bitstream to the bitstream parameter forward- 
ing path 728 and to the output interface module 721 . 

The output interface module 721 receives via the 
output bus 725 the bitstream of DCT coefficients output 
by the inverse quantizing module 707. The output inter- 
face module receives via the forwarded bitstream 
parameter bus 736 a bitstream composed of the 
respective picture headers and motion vectors from the 
input interface module 719. The output interface module 
forms the output bitstream 716 from the DCT coeffi- 
cients and the respective picture headers and motion 
vectors. 

The bitstream parameters derived from the input 
bitstream 712 by the first compression state translation 
unit 71 4A are forwarded via the bitstream forwarding 
path 728 to the first and second processors 71 8A and 
71 8B for use in processing the output bitstream gener- 
ated by the compression state translation unit 71 4 A. 
The bitstream parameters are also forwarded via the bit- 
stream forwarding path 728 to the compression state 
translation units 714B and 724 for possible use in re- 
encoding the modified bitstreams generated by the first 
and second processors. 

A second practical example of a compression state . 
translation unit will now be described. In this example, 
the compression state translation unit is again the first 
compression state translation unit 71 4A of a com- 
pressed bitstream processing apparatus 700 in which 
the first processor 71 8A operates on a partially-com- 
pressed bitstream composed of DCT coefficients. In this 
second example, the input bitstream 712 is a partially 
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compressed bitstream composed of pictures in coding 
order, and in which each picture is composed of a set of 
encoding blocks. Encoding blocks are picture blocks 
composed of a pixel value for each pixel when the pic- 
ture is a l-picture, and are difference blocks composed 5 
of a difference value for each pixel when the picture is a 
P- or B-picture. In the latter case, each encoding block 
has a respective motion vector. The compression state 
translation unit operates 71 4A to encode the partially- 
compressed input brtstrearn to change the compression 7, 
state of the input brtstrearn to that required by the first 
processor. 

Control data received via the control path 732 
cause the compression state translation unit 71 4A to 
operate in the following way: the orthogonal transform 7 
module 709 operates in its forward transform mode, and 
the remaining encoding/decoding modules are disa- 
bled. No computational resources are allocated to the 
disabled encoding/decoding modules. 

The input interface module 719 derives the encod- £ 
ing blocks, motion vectors and other brtstrearn parame- 
ters from the input bitstream. Using the input bus 725, 
the input interface module feeds the encoding blocks to 
the orthogonal transform module 709. The input inter- 
face module also feeds the motion vectors and other bit- 25 
stream parameters derived from the input bitstream to 
the bitstream parameter forwarding bus 736 for forward- 
ing to the bitstream parameter forwarding path 728 and 
to the output interface module 721 . 

The output interface module 721 receives via the 30 
output bus 725 the bitstream of DCT coefficients output 
by the orthogonal transform module 709. The output 
interface module also receives via the bitstream param- 
eter forwarding bus a bitstream composed of the 
respective motion vectors for the DCT coefficients from 35 
the input interface module 719. The output interface 
module forms the output bitstream 716 from the DCT 
coefficients and respective motion vectors. 

Again, the bitstream parameters derived from the 
input bitstream 7 1 2 by the first compression state trans- 40 
lation unit 71 4A are forwarded via the bitstream forward- 
ing path 728 to the first and second processors 71 8A 
and 718B for use in processing the output bitstream 
generated by the compression state translation unit 
71 4A. The bitstream parameters are also forwarded via 45 
the bitstream forwarding path 728 to the compression 
state translation units 71 4B and 724 for possible use in 
further changing the compression state of the modified 
brtstr earns generated by the first and second proces- 
sors. 50 

In both examples just described, forwarding the bit- 
stream parameters from the first compression state 
translation unit 71 4A saves the compression state 
translation units 71 4B and 724 from expending compu- 
tational resources to generate new parameters equiva- ss 2. 
lent to the bitstream parameters forwarded by the 
compression state translation unit 71 4 A. Consequently, 
the compressed bitstream processing apparatus 700 



according to the invention can perform the desired 
processing of the compressed input bitstream using a 
smaller scale of hardware or a less powerful computer 
or digital signal processor than a conventional com- 
pressed information signal processing apparatus. Alter- 
natively, compared with a conventional processing 
apparatus, the compressed bitstream processing appa- 
ratus 700 can perform the desired processing at a faster 
rate, e.g., faster than real-time, using given hardware or 
a given computer. 

Although this disclosure describes illustrative 
embodiments of the invention in detail, it is to be under- 
stood that the invention is not limited to the precise 
embodiments described, and that various modifications 
may be practiced within the scope of the invention 
defined by the appended claims 

Claims 

1. Apparatus (100, 700) for processing a compressed 
jt bitstream representing on information signal, 
apparatus comprising: 

a layered decoder (114; 714) that decodes the 
compressed input bitstream (1 12, 712) to pro- 
vide a decoded bitstream (1 16. 71 6A). the lay- 
ered decoder including serially-connected 
decoding modules (e.g., 201, 203. 705, 707), 
each having an output; 

a decoded bitstream processor (118, 71 8A) 
that processes the decoded bitstream in a way 
that modifies the information signal to provide a 
modified bitstream (122, 722A). the decoded 
bitstream processor having an input and an 
output; 

a layered encoder (124, 724) that re-encodes 
the modified bitstream to provide a com- 
pressed output bitstream (126, 726), the lay- 
ered encoder including serially-connected 
encoding modules (e.g., 301, 303; 705. 707), 
each of the encoding modules having an input, 
each of the encoding modules corresponding 
to one of the decoding modules of the layered 
decoder; and 

a selective interconnection path (134, 215, 
315, 138; 725, 721. 719, 738) that selectively 
connects the output of one of the decoding 
modules to the input of the decoded bitstream 
processor and that selectively connects the 
output of the decoded bitstream processor to 
the input of one of the encoding modules," the 
one of the encoding modules corresponding to 
the one of the decoding modules. 

The apparatus of claim 1 . additionally comprising 
means (120, 132, 142; 720. 732) for deactivating 
the decoding modules downstream of the one of 
the decoding modules, and for deactivating the 
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encoding modules upstream of the one of the 
encoding modules. 

3. The apparatus of claim 1 or 2, in which: 

5 

the layered decoder includes deriving means 
(e.g.. 203; 712) for deriving bitstream parame- 
ters from the compressed input bitstream; 
the apparatus additionally comprises a bit- 
stream parameter forwarding path (128, 728) 10 
that interconnects the layered decoder to the 
layered encoder; and 

at least one of the encoding modules is con- 
nected to the parameter forwarding path and 
performs an encoding operation using the bit- is 
stream parameters derived from the com- 
pressed input bitstream by the deriving means. 

4. Apparatus (100, 700) for modifying bitstream char- 
acteristics of a compressed input bitstream repre- 20 
senting an information signal, the apparatus 
comprising: 

a controller (120, 720) that provides an output 
bitstream definition defining bitstream charac- 25 
teristics of a compressed output bitstream; 
a layered decoder (114, 71 4 A) that decodes 
the compressed input bitstream (112. 712) to 
provide a decoded bitstream (116, 71 6A). the 
layered encoder including deriving means 30 
(e.g., 203, 719) for deriving bitstream parame- 
ters from the compressed input bitstream; 
a bitstream parameter forwarding path (128, 
728) connected to receive the bitstream 
parameters from the layered decoder; 35 
a layered encoder (124, 724) connected to the 
bitstream parameter forwarding path and that 
re-uses the bitstream parameters derived from 
the compressed input bitstream by the deriving 
means to re-encode the decoded bitstream in 40 
accordance with the output bitstream definition 
to provide the compressed output bitstream. 

5. The apparatus of claim 4, in which the decoded bit- 
stream is partially encoded relative to the informa- 45 
tion signed. 

6. The apparatus of claim 4 or 5. in which: 

the apparatus additionally comprises a so 
decoded bitstream processor (118, 71 8A) con- 
nected to receive the decoded bitstream (116, 
716) and to provide a modified bitstream (122, 
722A) to the layered encoder in lieu of the 
decoded bitstream (116. 716). the decoded bit- ss 
stream processor processing the decoded bit- 
stream in a way that modifies the information 
signal; and 



the layered encoder re-uses the bitstream 
parameters to re-encode the modified bit- 
stream. 

7. The apparatus of claim 6, in which: ~ 

the decoded bitstream processor is additionally 
connected to the bitstream parameter forward- 
ing path (128. 728); and 
the decoded bitstream processor processes 
the decoded bitstream in response to the bit- 
stream parameters derived from the com- 
pressed input bitstream by the deriving means. 

8. A method for processing a compressed input bit- 
stream representing an information signal, the 
method comprising: 

partially decoding the compressed input bit- 
stream (112) to provide a decoded bitstream 
(116), the decoded bitstream being a partially- 
encoded bitstream representing the informa- 
tion signal; 

generating a modified bitstream (122) by 
processing the decoded bitstream in a way that 
modifies the information signal, the modified 
bitstream being a partially-encoded bitstream; 
and 

re-encoding the modified bitstream (122) to 
generate a compressed output bitstream (126). 

9. The method of claim 8, in which: 

the method additionally comprises deriving bit- 
stream parameters from the compressed input 
bitstream; and 

the modified bitstream (122) is re-encoded 
using the bitstream parameters derived from 
the compressed input bitstream. 

10. A method for modifying the bitstream characteris- 
tics of a compressed input bitstream representing 
an information signal, the method comprising: 

generating an output bitstream definition defin- 
ing bitstream characteristics of a compressed 
output bitstream; 

deriving bitstream parameters from the com- 
pressed input bitstream (1 12); 
at least partially decoding the compressed 
input bitstream to provide a decoded bitstream, 
(116); and 

re-encoding the decoded bitstream using the 
bitstream parameters derived from the com- 
pressed input bitstream to generate the com- 
pressed output bitstream (126) having the 
bitstream characteristics def ined by the output 
bitstream definition. 
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11. The method of claim 10, in which the bitstream 
parameters derived from the compressed input bit- 
stream are bitstream parameters generated using 
large computational resources. 

12. The method of claim 10 or 1 1 , in which: 

the method additionally comprises processing 
the decoded bitstream (116) to generate a 
modified bitstream (122), the decoded bit- 
stream being processed in a way that modifies 
the information signal; and 
the modified bitstream is re-encoded in lieu of 
the decoded bitstream to generate the com- 
pressed output bitstream, the modified bit- 
stream being re-encoded using the bitstream 
parameters. 

13. The method of claim 10. 11 or 12, in which the 
decoded bitstream is processed in response to the 
bitstream parameters. 

14. Apparatus (700) for processing a compressed input 
bitstream representing an information signal, the 
compressed input bitstream having a compression 
state, the apparatus comprising: 

a first compression state translation unit (71 4A) 
that receives the compressed input bitstream 
(712) as an input bitstream and changes the 
compression state of the compressed input bit- 
stream to generate an output bitstream (71 6A); 
a processor (71 8A) that receives the output bit- 
stream of the first compression state transla- 
tion unit, processes the output bitstream in a 
way that modifies the information signal, and 
provides a modified bitstream; and 
a second compression state translation unit 
(724) that receives the modified bitstream as 
an input bitstream and changes the compres- 
sion state of the modif ied bitstream to generate 
an output bitstream (726), the first and second 
compression state translation units each 
including: 

layered encoding/decoding modules (e.g., 
705, 707, 709. 711), each having an input 
and an output 

an input interface module (719) that selec- 
tively connects the input bitstream to one 
of the encoding/decoding modules, the 
one of the encoding/decoding modules 
depending on the compression state of the 
input bitstream, and 

an output interface module (721) that 
selectively receives a bitstream from the 
output of one of the encoding/decoding 
modules as the output bitstream, the one 



of the encoding/decoding modules 
depending on the compression state of the 
output bitstream. 

5 15. The apparatus of claim 14, additionally comprising 
means for deactivating ones of the encod- 
ing/decoding modules not required to change the 
compression state of the input bitstream 

10 16. The apparatus of claim 14 or 15. in which: 

the first compression state translation unit 
includes means for deriving bitstream parame- 
ters from the input bitstream; 
is the apparatus additionally comprises a param- 

eter forwarding path (728) that interconnects 
the f irst compression state translation unit and 
the second compression state translation unit; 
and 

20 at least one of the encoding/decoding modules 

of the second compression state translation 
unit is connected to the parameter forwarding 
path and performs an encoding operation using 
the bitstream parameters derived from the 

25 compressed input bitstream. 
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17. A method for processing a compressed input bit- 
stream representing an information signal, the com- 
pressed input bitstream (712) having a 
30 compression state, the method comprising: 



providing layered encoding/decoding modules 
(e.g., 705. 707.709, 711); 
selecting a first at of encoding/decoding mod- 
ules and a second set of encoding/decoding 
modules from the layered encoding modules; 
changing the compression state of the com- 
pressed input bitstream (712) using the first set 
of encoding/decoding modules to provide a first 
output bitstream (71 6A); 
processing the first output bitstream to gener- 
ate a modified bitstream (722A), the first output 
bitstream being processed by in a way that 
modifies the information signal but generates 
the modified bitstream in the same compres- 
sion state as the first output bitstream; and 
changing the compression state of the modified 
bitstream using the second set of encod- 
ing/decoding modules to generate a second 
output bitstream (e.g., 71 6B, 726). 



40 



45 



SO 



55 



1a The method of claim 17, additionally comprising: 

selecting one of an encoding mode and a 
decoding mode for the encoding/decoding 
modules of the first set; and 
selecting one of an encoding mode and a 
decoding mode for encoding/decoding mod- 
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tiles of the second at 

1 9, The method of claim 1 7 or 1 8, in which: 

the method additionally comprises deriving bit- s 
stream parameters from the compressed input 
bitstream; and 

the compression state of the modified bit- 
stream is changed using the bitstream parame- 
ters derived from the compressed input 10 
bitstream 

20. The method of claim 1 7. 18 or 19. in which: 

the method additionally comprises deriving bit- 75 
stream parameters from the compressed input 
bitstream; and 

the first output bitstream is processed using the 
bitstream parameters derived from the com- 
pressed input bitstream. 20 
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(54) Method and apparatus for processing a bitstream 



(57) Apparatus (100, 700) for processing a com- 
pressed input bitstream (112, 712) representing an in- 
formation signal. The apparatus processes the com- 
pressed input bitstream in a way that changes the infor- 
mation signal represented by the compressed input bit- 
stream, and comprises a layered decoder (114, 714A) 
that decodes the compressed input bitstream to provide 
a decoded bitstream (116, 71 6 A), a decoded bitstream 
processor (118, 71 8A) that processes the decoded bit- 
stream to provide a modified bitstream (122, 722A), a 
layered encoder (124, 724) that re-encodes the modi- 
fied bitstream to provide a compressed output bitstream 
(126, 726), and a selective interconnection path (134, 
21 5, 31 5, 1 38; 725, 721 ,719, 738). The layered decoder 
includes serially-connected decoding modules (e.g., 
201, 203, 705, 707). The decoded bitstream processor 
processes the decoded bitstream in a way that modifies 
the information signal. The layered encoder includes se- 
rially-connected encoding modules (e.g., 301 , 303, 705, 
707), each of which corresponds to one of the decoding 
modules in the layered decoder. The selective intercon- 
nection path selectively connects the output of one of 
the decoding modules to the input of the decoded bit- 
stream processor and selectively connects the output of 
the decoded bitstream processor to the input of the one 
of the encoding modules corresponding to the one of 
the decoding modules. The layered decoder may in- 
clude an element that derives bitstream parameters 
from the compressed input bitstream, and the apparatus 
may additionally comprise a bitstream parameter for- 



warding path that interconnects the layered decoder to 
the layered encoder. At least one of the encoding mod- 
ules may be connected to the parameter forwarding 
path and may perform an encoding operation using the 
bitstream parameters derived from the compressed in- 
put bitstream. 



DECODING 
00NTRI 
DATA 



U4U 



132 



120 



processing 
selection 



112. 



7 



LAYERED 
DECODED 



COMPfiESSED 

INPUT 
BITSTREAM 



ENCODING 
CONTROL 
DATA 



1 



na 



BITSTREAM 
PROCESS08 



DECODED 
BITSTREAM 



125 



124 



LAYERED 
ENCODER 



«1 



MODlflE D 
BITSTREAM 



BITSTHEAM PAHAMETEflFOflWARBJNG 



COMPRESSED 

OUTPUT 
BTTSTREAM. 



430 



FIG.2 



Printed by Jouve, 75001 PARIS (FR) 



EP 0 889 650 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 98 10 6511 



DOCUMENTS CONSIDERED TO BE RELEVANT 




Category 


Citation of document with indication, where appropriate, 
of relevant passages 


Relevant 
to claim 


CLASSIFICATION OF THE 
APPLICATION (lntCI.6) 


Y 
X 


WO 95 35628 A (DEVLIN BRUCE FAIRBAIRN 
:KNEE MICHAEL JAMES (GB); SNELL & WILCOX 

* abstract * 

* page 16, line 1 - line 27 * 

* figures 1,10 * 


1-3, 
14-20 

4-13 


H04N7/26 
H04N7/5O 


Y 


PATENT ABSTRACTS OF JAPAN 
vol. 1996, no. 04, 
30 April 1996 (1996-04-39) 
& JP 07 336636 A (SONY CORP), 
22 December 1995 (1995-12-22) 

* abstract * 

& US 5 790 556 A (MATSUM0T0) 
4 August 1998 (1998-08-04) 

* column 2, line 31 - line 43 * 

* figures 4,5 * 


14-20 




X 


EP 0 696 873 A (AT & T CORP) 
14 February 1996 (1996-02-14) 

* abstract * 

* column 10, line 13 - line 19 * 

* figure 3 * 


4-13 




TECHNICAL FIELDS 
SEARCHED (lnLCt.6) 






HG4N 


X 
A 


EP 0 69Q 392 A (PHILIPS ELECTRONICS NV) 
3 January 1996 (1996-01-03) 

* abstract * 

* page 3, line 8 - line 16 * 

* figure 4 * 


4 

5-13 






W0 97 40626 A (ELECTR0CRAFT LAB LIMITED 
;KNEE MICHAEL JAMES (GB)) 
30 October 1997 (1997-10-30) 
* the whole document * 


1-20 




The present search report has been drawn up for all claims 







Place ol search 

THE HAGUE 



Dale of completion ol Ibe search 

22 March 2004 



Wahrenberg, A, 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant ff taken atone 

Y : particularly relevant V combined win another 

dooument of the same category 
A : technological background 
O : ncn-written disclosure 
P : tntei me d iate document 



T : theory or princpte underlying the invention 
E : earlier patent document, but published on, or 

after the fifing date 
D : document etted in the application 
L : dooument cited for other reasons 



& : member of the same patent tanttty, corresponding 
document 



2 



0889650A3 I > 



EP 0 889 650 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 
ON EUROPEAN PATENT APPLICATION NO. 



EP 98 10 6511 



This annex lists the patent family members relating to the patent documents cited in the above-mentioned European search report. 
The members are as contained in the European Patent Office EDP file on 

The European Patent Office is in no way liable (or these particulars which are merely given for the purpose of information. 

22-03-2004 



Patent document 
cited in search report 



Publication 
date 



Patent family 
memberfs) 



Publication 
date 



W0 9535628 



28-12-1995 



AU 2744095 A 

AU 3798199 A 

CA 2193109 Al 

DE 69529579 Dl 

EP 0765576 Al 

EP 0845908 A2 

W0 9535628 Al 

JP 10503895 T 

US 6285716 Bl 

US 2001631009 Al 



15-01 
26-08 
28-12. 
13-03 

02- 04 

03- 06 
28-12 
07-04 

04- 09 
18-10 



•1996 
■1999 
■1995 
■2003 
•1997 
■ 1998 
-1995 
-1998 
-2001 
-2001 



JP 07336636 



22-12-1995 



JP 
US 



3161228 B2 
5790556 A 



25-04-2001 
04-08-1998 



EP 0696873 



14-02-1996 



US 
CA 
EP 
JP 



5512953 A 
2152330 Al 
0696873 A2 
8070440 A 



3G-04-1996 
10-02-1996 
14-02-1996 
12-03-1996 



EP 0690392 



03-01-1996 



FR 2722052 Al 

FR 2724280 Al 

DE 69522861 Dl 

DE 69522861 T2 

EP 0690392 Al 

JP 8051631 A 

US 5729293 A 



05-01-1996 
08-03-1996 
31-10-2001 
11-04-2002 
03-01-1996 
2G-02-1996 
17-03-1998 



W0 9740626 



30-10-1997 



AU 
W0 

US 



2395597 A 
9740626 Al 
6323904 Bl 



12-11-1997 
30-10-1997 
27-11-2001 



2 , 

S! 

o 

a 



& For more details about this annex : see Official Journal of the European Patent Office, No. 1 2/B2 



noooccnto I ^ 



